Neural networks with learning and expression capability

ABSTRACT

A neural network comprising a plurality of neurons in which any one of the plurality of neurons is able to associate with itself or another neuron in the plurality of neurons via active connections to a further neuron in the plurality of neurons.

FIELD OF INVENTION

This invention relates to neural networks and particularly, though notexclusively, to neural networks based on one or more characteristicsincluding temporal, spatial, intensity, magnitude, and relativeposition; and may be used for one or more of: learning, knowledgeacquisition, discovery, data mining and expression.

BACKGROUND OF THE INVENTION

Existing neural networks are typically based on a single interpretationof Hebbian learning. This basic, Hebbian concept is often stated as“Neurons that fire together wire together”. The defacto interpretationis that wiring together is effected via the synapse that connects thetwo neurons together. The strength of the connecting synapse is modifiedor weighted to reflect the importance/probability of the presynapticneuron firing concurrently with the postsynaptic neuron, or vice versa.

Using the concept, neural networks have been developed that associate anumber of input neurons to a number of output neurons via synapses. Theinput neurons define the input states; and the output neurons define thedesired output states.

Thus nearly all existing neural networks are based on the concept ofthree layers: an input neuron layer, a hidden neuron layer, and anoutput neuron layer. FIG. 1 and FIG. 2 are illustrations of existingneural networks.

Training of such neural networks is accomplished, in its most basicform, by applying a specific input state to all the input neurons,selecting a specific output neuron to represent that input state, andadjusting the synaptic strengths or weights in the hidden layer. Thatis, training is conducted assuming knowledge of the desired output.After training has been completed, the application of different inputstates will result in different output neurons being activated withdifferent levels of confidence. Thus recognition of an input eventdepends on how close the original training states match the currentinput state.

Such neural networks typically require extensive, repetitive trainingwith hundreds or thousands of different input states, depending on thenumber of desired output neurons and the accuracy of the desired result.This results in practical networks of the order of only 10,000 input andoutput neurons with as many as 10 million interconnecting synapses orweights representing synapses (current existing neural networks are verysmall in size as compared to the capacity of the human brain which has10¹² neurons, and 10¹⁶ synaptic connections).

Furthermore, existing networks are trained on the basis of generatingpredefined output neurons, and can subsequently recognise inputs thatclosely resemble the training sets used for input. Existing neuralnetworks are not capable of independent learning as they are trainedusing prior assumptions—the desired goals are represented by the outputneurons. Existing neural networks are not capable of expressing orrecollecting an input state based on the stimulus of any output neuronin the output layer.

Existing neural networks are trained on the basis of applyingindependent input states, to the network, in which the order of trainingis typically insignificant. On completion of extensive, repetitivetraining, the output neurons are not significantly dependent on theorder in which input states are applied to the network. Existing neuralnetworks provide outputs that are based entirely on the current inputstate. The order in which input states are applied has no bearing on thenetwork's ability to recognise them.

Existing neural networks may have some or all of the followingshortcomings:

-   1. they require prior training, based on predetermined or desired    output goals—they do not learn;-   2. they can only recognise input states (objects) similar to the    input states for which they have been trained;-   3. they are highly computational, and therefore slow;-   4. they are computationally restricted to represent only a    relatively small number of neurons;-   6. they need retraining if they are to recognise different objects;-   7. they cannot express or recall an input object by applying a    stimulus to the output neurons;-   8. they are based on concurrent stimuli of all input neurons;-   9. they are not creative and they cannot express or recollect    events; they can only identify/recognise events for which they have    been trained;-   10. they assume neurons that fire concurrently or in quick    succession, are linked synaptically but do not distinguish one from    the other or the order of neuron firing; and-   11. each hidden layer neuron can receive inputs from multiple input    neurons concurrently.

SUMMARY OF THE INVENTION

In accordance with one aspect, there is provided a neural networkcomprising a plurality of neurons in which any one of the plurality ofneurons is able to associate or associate with itself or any otherneuron in the plurality of neurons via active connections to a furtherneuron in the plurality of neurons. This process is referred to aslearning.

In accordance with a second aspect there is provided a neural networkcomprising a plurality of elemental neurons, and a plurality ofstructural neurons for representing associations between any pair ofneurons, the pair of neurons being selected from the group consistingof: both elemental neurons, both structural neurons, one structural andone elemental neuron, and one elemental neuron and one structuralneuron.

Each structural neuron may represent the combined information or memoryrepresented by a pair of neurons. The process of recalling the pair ofneurons that were combined to form a structural neuron is referred to asexpression. Each structural neuron may receive input from only twoneurons.

The plurality of elemental neurons may be represented in the root levelof the neural network structure; and each elemental neuron may representat least one of: an elemental stimulus, a defined elemental pattern, anda defined elemental data element. Each elemental neuron may representone or both of: basic input stimuli and output stimuli of informationbeing processed. Each elemental neuron may be an equivalent of a neuronin a brain, the neuron in the brain being selected from the groupconsisting of a sensor neuron, a motor neuron, an intracortical neuronand an intercortical neuron. The information represented by a neuron maybe memory, and the processing may be learning or expression.

The plurality of neuron associations may be represented in a pluralityof deeper neural levels. The number of levels in the plurality of deeperlevels may be determined by the extent of the memory or pattern to beprocessed or expressed, where a memory represents a plurality ofelemental neurons. The number of elemental neurons and structuralneurons required to represent the memory may be determined by the natureof the memory to be processed.

In accordance with a third aspect there is provided a neural networkcomprising a plurality of neurons linked by associations, allassociations of neurons in a level of the neural network that is thesame or deeper being able to be expressed.

A fourth aspect provides a neural network comprising a plurality ofneurons, each neuron being represented by a unique addressable node inan array.

A fifth aspect provides a neural network comprising a plurality ofneurons, each neuron being represented in its entirety by a single nodein an array.

A sixth aspect is a neural network comprising a plurality of nodes in anarray, each node in the array comprising pointers. Each pointer is adata element of the node that represents a unique address of a specificnode in the array, each address representing a neuron of a plurality ofneurons. Each pointer represents a synaptic connection.

A seventh aspect there is provided a neural network comprising aplurality of neurons in an array, there being pointers in each node ofthe array for providing expression and for learning of memories.

A penultimate aspect provides a neural network comprising a plurality ofneurons, each neuron being represented by a node in an array, each nodehaving a plurality of pointers, each pointer in each node having aspecific and unique function. Except where a pointer may represent thevalue of an elemental stimulus in the elemental or root level neurons,each pointer contains an address of another neuron. The number ofpointers required may depend on the functions being performed by theneural network. For a neural network performing learning and expressionfunctions, the number of pointers needed will be at least four.

In this manner each neuron in the plurality of neurons may berepresented by a node of the same size in the array representing theplurality of neurons, each node containing a fixed number of pointers.

Nodes in an array used to represent neurons may also maintain additionaldata elements other than pointers pertaining to the characteristics ofeach neuron. Data elements may be defined to represent the frequency ofeach neuron's activation, the strength of its associations, and soforth.

The present invention also extends to a computer usable mediumcomprising a computer program code configured to cause one or moreprocessors to execute one or more functions to perform the methodsdescribed above.

In a final aspect there is provided a neural network wherein the neuralnetwork is bi-directional and is enabled to operate in a forwarddirection where nodes are derived or created from input, and in areverse direction where Input is derived from nodes. The forwarddirection is learning and the reverse direction is expression.

The neural network may be used for one or more of: monitoring andpredicting stock price movements, Internet surveillance, Internetsecurity, computer virus and spam detection, data compression, phraserecognition in speech and text, clauses in speech and text, plagiarismdetection, bioinformatics, vision recognition, semantic analysis andrepresentation of ontologies, and robotics.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the invention may be readily understood and put intopractical effect there shall now be described by way of non-limitativeexample only preferred embodiments of the present invention, thedescription being with reference to the accompanying illustrativedrawings in which:

FIG. 1 is an illustration of an existing neural network structure;

FIG. 2 is a further illustration of an existing neural networkstructure;

FIG. 3 is an illustration of an existing relationship between twoneurons and a synapse;

FIG. 4 is an illustration of the relationship between three neuronsaccording to the present invention;

FIG. 5 is a flow chart of the process flow of the present invention;

FIG. 6 is an illustration of the relationship between neurons andpointers/links;

FIG. 7 is an illustration of a sequence of events to illustrateassociation;

FIG. 8 is a flow chart for the learning process;

FIG. 9 is a flow chart for the matching process;

FIG. 10 is a flow chart for creating neurons during the learningprocess; and

FIG. 11 is a flow chart of the process for expressing of neurons.

DESCRIPTION OF PREFERRED EMBODIMENTS

According to a preferred aspect the present invention provides neuralnetworks, and a method for constructing such neural networks via neuronassociations, that are based on characteristics that Include at leastone of temporal, spatial, intensity, magnitude, and relative position,for the formation of memories, that consist of one or both of eitherinput stimuli (represented by elemental neurons) or output actions(represented by elemental neurons) in a natural manner.

It also provides for either or both of memory recollection and memoryexpression of one or more of the memories represented by structuralneurons, which represent multiple elemental stimuli. The neural networkallows for the potential expression of new actions or ideas other thanwhat It has learnt and in such a manner may exhibit creativity. Inputstimuli may include one or more of: audio, visual, tactile, and soforth. Output stimuli may include one or more of: movement, motion,speech, and so forth, each defined by appropriate elemental neurons.

Existing neural networks are based on the assumption that concurrentlyactivating two neurons (neurons B and C) creates an active synapticconnection between them, or strengthens existing synaptic connections.This is illustrated in FIG. 3 where there are two neurons and onesynapse.

Accordingly to one aspect of the present invention, stimulating oractivating two neurons creates an association between them via anotherthird neuron; the associating neuron. This is illustrated in FIG. 4where there are three neurons 41, 42 and 43 and two synapses 44 and 45.For convenience, this basic neural structure will be called a “neuronalassembly” throughout this specification. Neurons 41, 42 and 43 may beassociated together based on proximal characteristics, Including atleast one of temporal, spatial, intensity, magnitude and relativeposition. Neuron 43 will be at a deeper level within the neuralstructure than both of neurons 41, 42. Neurons 41,42 may be in the samelevel, or may be in different levels of the neural structure. The depthor level of a neuron in a neural network structure is based on thenumber of steps required to express the elemental neurons that itrepresents.

The neural structure comprises neurons, where each neuron represents amemory of data, events, objects, concepts or actions. The type ofinformation represented by each neuron can vary, and is dependent on theelemental neurons (representing sensor and/or motor neuron stimuli) fromwhich the neural network is constructed. Elemental stimuli are onlyrepresented in the elemental neurons maintained at the root levels ofevery neural network structure. Deeper or subsequent level neurons(structural neurons) only represent the association of other neurons anddo not in themselves store sensor, motor or elemental stimulus values.

Each neuron in the neural structure may represent the association ofonly two neurons, one an initiating neuron and the other an associatedneuron, although each neuron may participate as an initiating neuronand/or as an associated neuron in an unlimited number of associations,via associating neurons. An initiating neuron 41 can have any number ofsuccessor neurons such as neuron 43, where a successor neuron to neuron41 is a associating neuron (43) that has neuron 41 as its initiatingneuron. Another neuron 42 can have any number of precessor neurons,where a precessor neuron to neuron 42 is a associating neuron (43) thathas neuron 42 as its associated neuron. Thus, neuron 43 can be referredto as an associating neuron, or a successor neuron to neuron 41, or as aprecessor neuron to neuron 42.

The association is by one of the elemental neurons 41, 42 being aninitiating neuron and one is an associated neuron. Assuming neuron 41 isthe initiating neuron and thus neuron 42 is the associated neuron, whenneuron 41 Is activated or fires associating neuron 43 is potentiated. Atthe same time as, or subsequent to neuron 41 firing neuron 42 isactivated or fires and also potentiates associating neuron 43. Neuron 43is then considered activated. If the associating neuron 43 was nonexistent (that is there existed no neuron associating the initiatingneuron 41 and the associated neuron 42) then it is created and may bethen activated, otherwise it is only activated. The proximal activationor firing of neurons 41 and 42 causes the activation of associatingneuron 43, and the creation of active connections, or the strengtheningof existing synaptic connections, between neurons 41 and 43 and neurons42 and 43. The associating neuron 43 represents the sum of what islearnt from the other two neurons 41, 42. This sum may include one ormore of a memory trace, a combination of the experience of the two, asequence of events, a proximity of events and so forth. Once aassociating neuron is activated or created to represent a desired memoryor events, the desired memory or events need not be recreated in anotherneuron.

The definition of proximity or proximal activation or firing will be setby the rules of operation for each neural network. Proximity (“ΔP”) mayvary across a level, and across levels, and may vary according to thetype of elemental neuron being represented. Neurons at deeper levelswithin the neural network are less likely to be in close proximity, thusit is likely that ΔP will increase. If ΔP represents time or temporalevents ΔP may be as short as zero, milliseconds or seconds, or as longas minutes, hours, days or weeks. Events that occur simultaneously inthe real world may still be represented temporally based on differencesIn synaptic activation or firing times or the length of dendritic firingpaths. It may also vary according to the processing requirements of thememories or events being represented by the neural structure. ΔP mayalso represent spatial relations between objects or events and can alsorepresent the topological spatial relationships of neurons in the brain.

A neural network according to the present invention consists of twobasic elements:

-   -   1. Elemental neurons, which can represent elemental stimuli,        sensor or motor neurons or such elemental data as required or        desired. These are the elemental or root neurons from which a        neural network is constructed. Different types of elemental        neurons may be defined depending on the type of experience or        events or information being represented. For example if        representing the auditory cortex the elemental neurons would be        for representing distinct sounds. Elemental neurons may        incorporate a number of different types of elemental neurons        such as, for example, one set for representing sound energy or        intensity of the sound (volume), and another set for        representing the frequency of the sound.    -   If representing smell and taste there may be a set of elemental        neurons for taste and another set for olfactory sensing.    -   In vision there may be sets of elemental neurons to represent        the colour cones, rods, edges, contrast, movement, and so forth,        as represented by the ganglion cells, or specifically to        represent the photoreceptor neurons.    -   For skin receptors there may be elemental neurons corresponding        to touch—mechanoreceptor neurons; temperature—thermo receptor        neurons; pain—nociceptor neurons; and so forth. Motion can be        represented by the various types of motor neurons that induce        movement.    -   When a sensory neuron is activated or fires, it communicates to        different areas of the brain via synapses or nerve cells that a        certain form of energy from a specific cell or sensory organ at        a specific location has been received. All sensory neurons have        similar central processing mechanisms. When a motor neuron is        activated or fired in the brain it induces muscle contraction at        a specific location in the body thus producing movement. When a        sensory neuron detects an event it passes the data to the brain        where it is processed in the brain's neural structure.    -   Artificial elemental neurons may also be defined. For example,        if using the neural structure to process English text, a set of        elemental neurons may be defined to represent the alphabetic        characters and punctuation characters. For the Chinese language,        elemental neurons could be defined for each Chinese character,        or a subset thereof. If using the neural structure to process        protein sequences, the elemental neurons may be defined to        represent the twenty amino acids, or subsets thereof. For        speech, different motor neurons may be defined to produce        different muscular contractions resulting in the production of        sounds corresponding to phonemes, and so forth.    -   Elemental neurons can be initiating and associated neurons but        cannot be a associating neuron.    -   2. Structural neurons representing the neural structure. The        neural structure as illustrated in FIG. 4 consists of neurons        that represent the association of other neurons, whether they        are sensor neurons, motor neurons, or other structural neurons.        In this manner neural structures can grow, representing more and        more information.    -   Structural neurons can also be used to form associations between        structural neurons representing different sets of elemental        neurons. For example, an association may be formed between the        English word “ball” represented by an association of alphabetic        elemental neurons, with the shape “ball” represented by an        association of visual elemental neurons. In this manner it is        possible to build neural networks which allow the association of        information across different cortexes or across different neural        networks. Another association between the shape “ball” may exist        with the spoken word “ball” represented by an association of        motor neurons to enable phoneme neurons to produce the sound        “ball”.    -   Structural neurons are associated with each other on the basis        of characteristics including temporal, spatial, intensity,        magnitude and relative position. If representing speech with the        neural structure, the associations would be of a temporal        nature, representing the sequence of phonemes, words, phrases,        clauses, and so forth, used in expressing speech. Likewise, if        processing text or reading a book the processing of individual        characters would be of a temporal nature building up the words,        phrases, clauses, and so forth. Many neurons in the brain are        also organised spatially or topographically, such as those for        vision, hearing, touch, pain, and so forth. As such it is        possible to construct neural structures that represent spatial        characteristics. For example, in forming a neural structure        representing a visual scene, neurons representing edges or lines        or curves or objects or patterns may be associated into a neural        structure that associates elemental neurons representing pixels        into lines or curves based on proximity, near objects with far        objects, or high objects with low objects, thus building a three        dimensional map of the environment.    -   The same neural structure can be used to represent information        in any of the four dimensions defined by space and time.    -   As above, the shape “ball”—represented by a spatial neural        structure—may be associated with the spoken word “ball”        represented by a temporal neural structure.    -   A structural neuron is a associating neuron. It can also be one        or both of an initiating neuron and an associated neuron.

The combination of elemental neurons at the root levels of associationwithin the neural structure allows for the creation of structuralneurons that represent particular elemental features or characteristicswithin a cortex (neuronal assemblies). The neural structure allows forthe representation of feature combinations by dynamic association withinthe neural structure. Neural activity in the neocortex that is evoked bysensory neurons is always distributed in the brain to different areas.In sensory systems, different areas of the brain can process differentaspects (spatial/temporal/intensity/magnitude/relative position) of thesame input stimuli. The proposed neural structure allows for intra-arealneural associations in order to represent coherent concepts/percepts andbehaviour. Deeper levels of association within the neural structure(intra cortico-cortical associations) allows for the representation ofincreasingly complex information or behaviour.

Neurons may be organised, classified or named according to theirfunctions, characteristics, levels, nature, and so forth. Neurons mayalso be defined for a particular dimension. That dimension may be time,distance, space, length, height, pitch, amplitude, or any otherdefinable characteristic.

Within its particular dimension, any two neurons may be differentiatedaccording to their relative or related occurrence, position, scale ormagnitude. This relative positioning is represented by ΔP.

The basic underlying parameter for constructing a neural network is thatfor any two active neurons A related to B by relative position thereexists a third neuron C that has connections via synapses to bothneurons A and B. If such a neuron doesn't exist then it is possible tocreate such a neuron and its connections. Therefore neuron C associatesneurons A and B together maintaining their relative positions. A and Bmay be, but do not have to be, contiguous or consecutive within theirrelative positions. Therefore neuron C can associate any two neurons Aand B which represent events whether the events are concurrent,contiguous, consecutive, non-contiguous, non-consecutive or overlapping.

The logic of the association is that for any two active neurons A and Bthat are activated or fire, one of them will be an initiating neuron.The third neuron C will associate the initiating neuron with the secondneuron preserving their order of activation or firing. As such, neuron Crepresents the combined events of the initiating neural event followedby the second neural event. This new neuron C can subsequentlyparticipate in combining with other neurons (thus creating newassociations), and so forth.

The basic rule for creating new neurons is that if neuron A is activatedor fired, concurrent with or followed by neuron B being activated orfired, then a new neuron C can be constructed and possibly activated orfired, representing the association of neurons A and B (i.e. theassociation of events AB in time or space). If neuron C already existsdue to a prior event or association then neuron C can likewise bere-activated and fired, allowing it to participate in subsequentexisting or new activations, thereby allowing it to form newassociations. This allows for the construction of neural networks withan unlimited numbers of associations and/or relationships.

A flowchart representing the basic flow for creating a new neuralnetwork is shown in FIG. 5. This preferred example shows the processingsteps and application of the model for processing a simple sequence ofelemental events.

In FIG. 5, the neural network or array representing the neural networkis initialized at 50. At 51 the type or category of each elementalneuron is defined and may be related to, for example, text, visionpitch, colour, edge, sound, volume, taste, olfactory sensing, colourcones, rods, pain, contrast, movement, muscular movement, ganglioncells, photoreceptors, touch, mechanoreceptors, temperature,thermo-receptor, nociceptor, motion, language, characters, letters,words, and so forth.

The elemental neurons are then defined/programmed/input at 52. Thequestion is then asked: does the elemental neuron already exist? (53).For each unique value to be represented for each defined type of definedelemental neuron, all elemental neurons must be created, and areattached as a list of successor neurons to the root neuron, the rootneuron being represented by node zero in the array representing theneural network. If the answer to the query is no, a new elemental neuronis created (54) and the process returns to (52). If yes, the processcontinues to 55. Here, the rules to associate neurons is defined.Alternatively, new elemental neurons may be defined as and when theyoccur during the learning process.

The first, or next initiating, neuron Is then found (56). This isdesignated neuron A. The subsequent neuron (neuron B) is then found(57). If an association of neuron A and neuron B already exists (58),the process reverts back to 56. If not, a neuron C is created asrepresenting the association or combination of neuron A and neuron B(59) and the process reverts back to 56. Neuron C can be referred to inthis process as the associating neuron.

FIG. 8 is a flow chart providing more detail of steps 56 to 59 of FIG.5. Here, neural processing or learning is of an experience representedby a pattern or sequence of elemental neuron events.

In step 81, the event pointer is set to the first elemental event in theevent pattern or sequence. A matching neuron is found in step 82. Thematching neuron is a neuron (memory) in the neural network that matchesthe leading event or events in the pattern or sequence of events pointedto by the event pointer. This matching neuron is defined as theinitiating neuron (neuron A). The event pointer is then updatedaccording to the defined rules for the neural network to point to thenext event.

Another matching neuron is then found in step 83. Again, this neuron(memory) is for the event or events in the pattern or sequence of eventspointed to by the event pointer. This another matching neuron is definedas the associated neuron (neuron B). The event pointer is then updatedaccording to the defined rules for the neural network to point to thenext event. In this manner new associating neurons or associations arealways created, subject to any rules to the contrary. Generally theneuron C could not have previously existed, otherwise it would have beenmatched at step 82.

Having completed the processing of a single pass, the same data or eventsequence may be processed again if so desired, extending the neuralnetwork structure accordingly.

A further neuron that has not been used is then found in step 84. Thisis created as the associating neuron (neuron C) in step 85. The processof steps 82 to 85 are repeated in 86 until there are no more events inthe pattern or sequence of events being processed.

FIG. 9 provides more detail of the process steps 82 and 83 of FIG. 8—thefinding of a matching neuron.

In step 91 the matching neuron pointer is set to zero by pointing to theroot neuron. For each successor neuron of the matching neuron (step 91),a check is conducted to determine if the complete expression of thesuccessor neuron matches the subsequent event or events or sequence inthe pattern or sequence of events being processed. If it does, theprocess moves to step 93 where the pointer for the matching neuron isset to point to the successor neuron that matches the event pattern orsequence. In addition, the event pointer may be adjusted to point to theremainder of the pattern or sequence of events that, as yet, have notbeen matched. The process then reverts to step 92 and continues. If theresult of step 92 is that it doesn't match, in step 94 the matchingneuron pointer is returned pointing to the last matching neuron, and theprocess ends in step 95.

FIG. 10 illustrates in more detail the process of step 85 in FIG. 8—thecreation of a associating neuron. In step 101, the initiating pointer ofthe associating neuron is set to the point to the initiating neuron, andin step 102 the associated neuron pointer of the associating neuron isset to the associated neuron. As the associating neuron has no successorneurons (having been just activated or created), its successor neuronpointer is set to zero (step 103). The next neuron pointer of theassociating neuron is then set to be equal to the successor neuronpointer of the initiating neuron. Finally, in step 105, the successorpointer of the initiating neuron is set to point to the associatingneuron and thus the associating neuron becomes the first entry in thelist of successor neurons to the initiating neuron.

A group of elemental neurons representing a common characteristic may beorganised into a cortex. Thus, it is possible to have a visual cortexcontaining groups of neurons which relate to a first visualcharacteristic (for example, a group of neurons for colour); a secondgroup for second visual characteristic (for example, shape); a thirdgroup for a third visual characteristic (for example, size), and soforth. Structural neurons from one neural network, representing distinctpatterns or events, can be elemental neurons in another neural network.Multiple cortexes, possibly represented by multiple neural networks (andthe neurons within them), may be interlinked and related to form anintegrated ‘brain’ which can provide not only an integrated learningenvironment, but also the potential for intelligent behaviour.

Traditional neural networks are typically unidirectional. This meansthat, given a certain input the output can be derived. However, theycannot work in the reverse direction. That is, given the output, it isnot possible to derive the input.

The present invention provides for the capability for ‘expression’,whereby the sequence of events that leads to the construction of anyneuron can be expressed. The importance of expression is that it allowsa complete sequence of events to be represented by a single neuron, andthat sequence can be reproduced by activating that single neuron and, inturn, the neurons that represent the association that is the initiatingneuron and the associated neuron in the same relative position or orderin which they were created, and so forth. In this manner it is possiblefor a single neuron representing a complete experience or memory(sequence of events) to be expressed, recalling the experience.

The expression ability is implemented by way of the initiating pointerand the associated pointer for each neuron in the neural structure.

Expression is illustrated in FIG. 11. When expression is required, thefirst check in step 111 is to determine if the initiating pointer of theneuron to be expressed is equal to zero. If yes, it must be an elementalneuron and the process reverts to step 114 where expression isactivation of the element event or value stored in the associated neuronto represent the elemental neuron. If the value is not zero, step 112 isstarted. In step 112 is expression of the neuron pointed to by theinitiating neuron pointer of the neuron to be expressed. In step 113 isexpression of the neuron pointed to by the associated neuron pointer ofthe neuron to be expressed. This continues until all elemental neuronsrepresenting the memory have been expressed.

Each neuron in the network has an address to identify its locationwithin an addressable array of nodes where each node represents a singleneuron. All neuron pointers refer to an address of a node representing asingle neuron in the array of neurons. The bit size of the address orpointer will influence the maximum number of neurons possible to berepresented in the array representing the neural network. For example:using a pointer address size of 32 bits will allow the construction ofneural networks consisting of 2³² or 4,294,967,296 neurons.

Each neuron is represented by a single fixed length node in aconventional array or structure. The number of neurons that may bestored in an array will be dependent on the total memory storageavailable (internal and/or external), and on the pointer addressarchitecture:

-   -   (a) a 16 bit pointer address architecture will allow for up to        2¹⁶ neurons. This is 6.4×10³ neurons;    -   (b) a 32 bit pointer address architecture will allow for up to        2³² neurons. This is 4.1×10⁹ neurons;    -   (c) a 48 bit pointer address architecture will allow for up to        2⁴⁸ neurons. This is 2.7×10¹⁴ neurons; and    -   (d) a 64 bit pointer address architecture will allow for up to        2⁶⁴ neurons. This is 1.8×10¹⁹ neurons.

As the human brain has approximately 10¹² neurons, it may be possiblematch the capacity of a hundred human brains using a 48 bit pointeraddress architecture. Based on the latest computers that support 64 bitaddressing, it may be possible to have the capacity to represent neuralstructures consisting of 10¹⁹ neurons, the equivalent size of 10 millionhuman brains wired together.

To represent the neural network, a node will consist of a minimum offour pointers. Each pointer contains an address to another neuron. Asexplained above, using an address size of 32 bits will allow theconstruction of neural networks consisting of up to 2³² or 4 billion,neurons.

As shown In FIG. 6, each neuron may be represented by fourpointers/links (P₁, P₂, P₃ and P₄), in the most basic implementation.Each pointer is a link to the address or location of another neuron.Typically, a neuron will have four pointers, the minimum needed tocreate an intelligent neural network with expression capabilities.However, a neuron may have more than four pointers, to provideadditional functionality. For example in creating neuron C to representthe association of an initiating neuron A and an associated neuron B,the following pointers are required for neuron C:

-   -   P1—a pointer to the initiating neuron, i.e. neuron A;    -   P2—a pointer to the other neuron that participates in forming        the association, i.e. neuron B;    -   P3—a pointer to a list (LsC) of deeper level neurons that has        neuron C as an initiating neuron (a successor pointer for neuron        C); and    -   P4—a pointer to the next neuron,(LA) that has neuron A as its        initiating neuron (a ‘next successor’ pointer for neuron A).

Additional pointers can also be defined for neuron C if necessary toprovide information on the precessors to the associated neuron. Forexample:

-   -   P5—a pointer to a list (LpC) of deeper level neurons that has        neuron C as an associated neuron (a precessor pointer); and    -   P6—a pointer to the next neuron in a list (LB) that has neuron B        as its associated neuron (a ‘next precessor’ pointer).

Deeper level neurons may represent complex sequences of events. Sinceeach structural neuron has at least one initiating neuron, and oneassociated neuron, a single neuron at level 10 may represent a sequenceof up to 2¹⁰ or 1,024 elemental events. It is not a necessary conditionthat neurons can only associate with neurons of the same level.

Thus, in storing new sequences of events it is only necessary toidentify those existing neurons which represent existing event sequencesin the event stream and associate these together by constructing newneurons, e.g. if we have two event sequences of say 1,024 events and 512events, respectively, it is possible to construct a single, new neuronto represent an event sequence of 1,536 events. Thus new sequence/eventinformation can be economically represented in the neural network.

This is illustrated in FIG. 7. FIG. 7 is an illustration of a possibleneural network derived from processing text and learning, for thesentence; “The cat sat on the mat.” An arrowed line indicates theconnection between an initiating neuron and a associating neuron, and acircled line indicates a connection between the associated neuron andthe associating neuron.

In FIG. 7( a), the first elemental neuron 701 and the second elementalneuron 702 in level zero recognise or represent the letters “T” and “H”and associate to form the letter combination “TH” represented by theassociating neuron 711 in level 1.

FIG. 7( b) shows the third elemental neuron 703 recognising the letter“E” and the fourth elemental neuron 704 recognising the space “^”. Thesetwo associate together to form the combination “E^” represented byassociating neuron 712 in level 1. The structural neurons 711 and 712 inlevel 1 associate to form the distinct word “THE^” represented by theassociating neuron 721.

In FIGS. 7( c), (d) and (e), the next two elemental neurons In levelzero—705 and 706—recognise the letters “C” and “A” respectively, andassociate to form the letter combination “CA” represented by associatingneuron 713 in level 1.

For the letters “T” and “^”, neuron 701 associates with neuron 704 tocreate a associating neuron 714 in level 1 representing the combination“T^”. Neurons 714 and 713 then associate to create a associating neuron722 in level 2 thus forming the distinct word “CAT^” (FIG. 7( e)).Neurons 721 and 722 can then associate to give a result at theassociating neuron 731 in level 3 to form the phrase “THE^ CAT^” (FIGS.7( f) and 7(g)).

For the word “SAT^”, as is shown in FIG. 7( h), neuron 707 recognisesthe letter “S” and associates with neuron 706 to give a result atassociating neuron 715 in level 1 for the letter combination “SA”.Neuron 715 associates with neuron 714 in level 1 (“T^”) to give a resultat associating neuron 723 in level 2 for the distinct word “SAT^”.

In FIGS. 7( i) and 7(j), neurons 708 and 709 recognise the letters “O”and “N” respectively and associate to form the letter combination (andword) “ON” represented by associating neuron 716 in level 1. Neuron 716associates with neuron 704 to give a result at associating neuron 724 inlevel 2 for the distinct word “ON^”. Neuron 723 in level 2 associateswith neuron 724 to give a result at associating neuron 732 in level 3for the clause “SAT^ON”. Neurons 731 and 732 in level 3 may associate togive a result at neuron 741 in level 4 for the clause “THE^CAT^SAT^ON^”.

FIGS. 7( k) and (l) Illustrate neuron 7010 recognising the letter “M”and associating with neuron 706 to give a result at neuron 717 in level1 for the letter combination “MA”. Neuron 717 associates with neuron 712(“T^”) to give a result at neuron 725 for the distinct word “MAT^”.Neuron 721 associates with neuron 725 (“THE^”) to give a result atneuron 733 for the phrase “THE^MAT^”.

Neurons 741 and 733 can associate to give a result at neuron 751 inlevel 5 for the sentence “THE^CAT^SAT^ON^THE^MAT”. It is clear that eachassociating neuron has at most one initiating neuron and one associatedneuron. However, neurons can have multiple successor and precessorneurons. For example elemental neuron 701 “T” has successor neuronsrepresenting “TH” (711) and representing “T^” (714). Elemental neuron706 “A” has three precessor neurons representing “CA” (713), “SA” (715)and “MA” (717).

To extend the sentence illustrated in FIG. 7( m) where the neuron 7011in level 0 recognises the letter “D”. Neuron 7011 associates with neuron704 to give a result at neuron 719 for the combination “D^”. Neuron 706associates with neuron 709 to give a result at neuron 720 for thecombination “AN”. Neuron 720 associates with neuron 719 to give a resultat neuron 726 for the combination “AND^”. Neuron 733 associates withneuron 726 to give a result at neuron 742 for the combination“THE^MAT^AND^”. Neurons 741 and 742 may the associate to give a resultat neuron 752 for the combination to represent the phrase“THE^CAT^SAT^ON^THE^MAT^AND^”

In FIG. 7( n) it shows that in level 0 each elemental neuron representsonly 1 letter/character. Each neuron in level 1 represents anassociation of 2 letters/characters; in level 2 each neuron mayrepresents an association of up to 4 letters/characters; in level 3 upto 8 letters characters; up to 16 letters/characters in level 4; up to32 letters/characters in level 5; and so forth in multiples of 2 untileach neuron in level 10 could represent the association of up to 1,024letters/characters; and each neuron in level 20 could represent theassociation of up to 1,048,576 elemental events.

As can be seen, for text recognition the number of elemental neuronsrequired is quite limited. One elemental neuron would be required foreach of:

-   1. the 26 letters of the alphabet upper case;-   2. the 26 letters of the alphabet lower case;-   3. the 10 numbers;-   4. the 32 punctuation and similar characters on the standard    alphanumeric keyboard;-   5. any other special characters (e.g. from the Greek alphabet) that    may be required.

The number of structural neurons required and the number of levels ofstructured neurons required can grow in consequence of use of the neuralnetwork, during the learning process. Using the example of textrecognition, the number of documents concerned, the size of the learningevent(s) concerned, the use of language in the documents, and themaximum possible number of words, phrases, clauses, sentences,paragraphs, chapters, books, and so forth will impact the size of theresulting neural network structure. By way of example, if all the playsof William Shakespeare were recorded using the present invention in textrecognition, after the first play was recorded there would be a certainnumber of structural neurons “N1”. When the second play was recorded,within the same neural network, the total neurons required would not bethe simple arithmetic sum of those required for the independentprocessing of each of the plays. Only the extra neurons required toprocess the second play above and beyond those created for the firstplay would be added to N1 to give the enlarged total N2. Neurons for anypreviously existing letters, words, phrases, clauses, sentences (and soforth) would not be added, but their new associations would be added.Upon the third play being recorded, the extra neurons required would beadded in the same way to give N3.

By the time the final play was entered, the increase in the number ofneurons required would not be great. The main increase would be atdeeper levels in the neuron structure where new associations would beadded. The deepest level could be a single neuron having allassociations required to represent in their entirety all the plays.

As each neuron in each level can associate with any other neuron in anylevel to give a result at a third neuron, the number of potentialpermutations and combinations available is very large.

From a consideration of FIG. 7, there are different combinations thatmay be used to provide the result at neuron 751“THE^CAT^SAT^ON^THE^MAT”. For example, in level 1 as illustrated thefollowing combinations are recorded:

-   -   “TH”, “E^”, “CA”, “T^”, “SA”, “ON”, and “MA”.

Other combinations are possible. For example:

-   -   “TH”, “HE”, “E^”, “CA”, “AT”, “T^”, “ON”, “N^”, and “MA”.

This can be followed by a further range of possible combinations inlevel 2. By adopting rules pertinent to a desired objective for usingthe neural network, to reduce or eliminate combinations, efficiency maybe enhanced.

If a level 10 neuron is considered, it is capable of representing asequence of up to 1,024 elementary events. The first element can beexpressed by following the pointers up to the elemental or root level.This allows a relatively speedy expression.

Elemental neurons may be frequently activated both for learning andduring expression. In a pure binary implementation two elemental neuronsor root level neurons can be expressed or associated by a single level 1neuron. Two level 1 neurons may be expressed or associated by a level 2neuron and so forth. If a sequence of 1,024 elemental events isactivated and learnt/expressed by the network then the following numberof neurons at each level may be processed:

1,024 level 0 512 level 1 256 level 2 128 level 3 64 level 4 32 level 516 level 6 8 level 7 4 level 8 2 level 9 1 level 10representing the entire sequence of elementary events.

If the basic time between neurons firing was 1 millisecond, at the rootlevel, a level 10 neuron would only activated or fired once every 1024milliseconds or once a second. Thus if the array representing the neuralstructure is represented as a combination of distinct memoryblocks/areas, deeper level neurons could be stored in slowerstorage/memory area without impacting the performance of learning andexpression.

In the preferred neural network structure, elemental neurons have astheir initiating neuron the root neuron for the neural networkstructure. This allows an unlimited number of elemental neurons to becreated on the fly, rather than having to predefine elemental neurons.Alternatively, elemental neurons can be allocated a predetermined neuralstorage/memory area in the structure, and each elemental neuron can thenbe directly addressed (e.g. for ASCII text characters 8 bits), 256elemental neuron addresses can be pre-allocated (0 to 255) and theaddress of the first structural neuron will be at address 256.

In the preferred mode, new elemental neurons can be added at anytime,anywhere in the neural structure, providing for complete flexibility.When using a predefined area for the elemental neurons, processing maybe faster as there is no need to search a list of successors attached tothe root neuron.

Elemental neurons may be represented with their initiating neuronpointer set to zero—pointing to the root neuron (whether it exists ornot), and their associated neuron pointer set to the value of theelemental event to be represented by that elemental neuron. A elementalneuron can always be determined or identified easily as its initiatingneuron value is always zero. Therefore, when expressing neurons it iseasy to determine when we have reached an elemental neuron because theinitiating neuron value is zero, pointing to the root neuron.Alternative methods may be used to define elemental neurons, withoutimpacting the essence of the neural network presented.

In the preferred mode, new successor neurons to an Initiating neuron aresimply added to the front of the list. Therefore, they are attacheddirectly to the initiating neuron. In this way recent memory traces arereadily expressed.

However, various options are available to maintain lists of successorneurons. The neural structure allows this feature to be used in anymanner the user chooses in order to provide added functionality to theneural network structure. For example, new associating neurons could beadded to the end of the list, or the list could be maintained innumerical order, alphabetical order, and so forth. Likewise, neuronswhich are frequently accessed could be moved towards the front of thelist, or at the front of the list, such that more recently activatedmemories are always more accessible. This also means that older memoriesare at the end of the list and less likely to experience expression.

In this manner the order of the list can be used to represent therelative synaptic strengths or activation levels of the successorneurons to an initiating neuron without having to use weights torepresent the strength of synaptic connections, if so desired.

The memory represented by any neuron can be expressed by simplyexpressing the neurons represented by the initiating neuron pointer andthen the expressing the neuron represented by the associated neuronpointer. If the initiating pointer is zero, it is possible to expressthe elemental value of the elemental neuron. Otherwise, the process canbe repeated for the initiating pointer—express its initiating neuronpointer and its associated neuron pointer, and so forth. Thus, anyneuron can be fully expressed by expressing its parts in order.

It is possible to create neural network structures representingknowledge learnt For example, if the following sentences of text areinputs:

-   -   LUCY^IS^CRYING^UNDER^THE^TABLE^.    -   JOHN^IS^PLAYING^IN^THE^PARK^.    -   PETER^IS^READING^IN^THE^CHAIR^.    -   MARY^IS^SLEEPING^UNDER^THE^TREE^.    -   JOHN^IS^RUNNING^IN^THE^RACE^.    -   PETER^IS^PLAYING^ON^THE^SWING^.    -   MARY^IS^TALKING^ON^THE^PHONE^,        it is possible to create neurons that represent the following        phrases or memories:    -   IS CRYING    -   IS RUNNING    -   IS READING    -   IS SLEEPING    -   IS PLAYING    -   IS TALKING

In this case “IS^” has six successor neurons.

Similarly it is possible to create the following phrases or memories:

-   -   UNDER THE TABLE    -   UNDER THE TREE    -   IN THE PARK    -   IN THE CHAIR    -   IN THE RACE    -   ON THE SWING    -   ON THE PHONE

Thus LUCY is associated with IS CRYING, but IS is associated with sixdifferent actions. By changing the expression of alternative successorneurons for IS, it is possible to express up to six differentphrases/clauses:

-   -   LUCY IS CRYING    -   LUCY IS RUNNING    -   LUCY IS READING    -   LUCY IS SLEEPING    -   LUCY IS PLAYING    -   LUCY IS TALKING

Although only the first phrase was ever learnt, or input to the neuralnetwork, it is possible to do the same for JOHN, PETER and MARY.

Effectively it is possible to generate new expressions based on thestructure **IS and IS**. That is:

LUCY IS IS CRYING JOHN IS IS RUNNING MARY IS IS READING LUCY IS ISSLEEPING PETER IS IS PLAYING IS TALKING

Therefore, it is possible to create or express 30 (5×6) phrases/clausesthat are all syntactically correct, and are also all semanticallycorrect. Similarly:

ON THE THE TABLE IN THE THE PARK UNDER THE THE CHAIR THE TREE THE RACETHE SWING THE PHONEwhich provides for the possibility of generating 21 phrases/clauses ofthis nature (3×7) which are all syntactically correct, but not allsemantically correct.

Combining the two sets together gives the structure:

-   -   {*} IS {*} (*) THE {*}.

This provides for the expression of {5} IS {6}*{3} THE {7}, or5×6×3×7=630 possible expressions, having learnt only seven inputsentences. Thus the neural model allows the learning and building aneural structure of memories, and also allows the combining of thoseneurons or memories into new expressions or behaviour, according to userdefined processes. The brain can explore new concepts or suggestionsthat are syntactically possible with what has previously been learnt.With millions of neurons, it is possible to express billions ofsyntactically correct sentences. That is, the neural network structureprovides an ideal method for creative expression.

By use of a neural network described above, it is possible to have asystem that:

-   -   is able to learn;    -   has expression capabilities;    -   stores associations rather than data;    -   has an efficient usage of computer memory and storage space; and    -   is computationally efficient.

Thus the network is able to recognise patterns within patterns ofassociations. As such it may be of use in varying industries such as,for example, monitoring and predicting stock price movements, internetsurveillance, security, computer virus detection, phrases in speech andtext, clauses in speech and text, plagiarism detection, datacompression, and so forth.

The neural network structure and processes described above may beimplemented in software or hardware. If in hardware, they may be part ofa chip, all of a dedicated chip, or an array of chips, all beingelemental and or structural neurons. Elemental and structural neuronsmay be in part of the chip, or may be in a memory array of dedicatedneurons. As all neurons exist, upon the first pass of data the firstelemental neuron will learn the first element of the data. For example,the upper case letter “T” in the example used in FIG. 7. The nextelemental neuron would learn the next element of the data. Again usingthe example of FIG. 7, the lower case letter “h”. And so forth. As theelemental neurons and those in the lower levels of the neural networkwill be activated more frequently to create the associations in thedeeper levels, they may require faster processor speeds, for bestperformance. Neurons at deeper levels will be activated less frequentlyand therefore could be stored in storage with slower processor speeds.

Therefore, there is provided a neural network based on the concepts ofneurons including sensor and motor neurons, and synaptic connections.Each neuron in the brain can be represented by a single node in an arrayor memory in a simple manner. Neurons are fixed-length nodes in an arrayor memory. Each synaptic connection can be represented by pointers toother neurons within each node. Neural nodes in the array may consistsolely and exclusively of pointers to other neurons in the data array.However, motor or sensor neurons contain a sensor value or motor valuefor interacting with the external environment. Each neuron containsconnections to other neurons. If desired, neurons may also maintainother information such as, for example, the frequency of activationduring learning, the number successor neurons, the number of precessorneurons, and so forth, to assist processing and to be used in the rulesfor creating associations, and for expression.

Elemental neurons, corresponding to sensory or motor neurons can bedefined for interfacing to and interacting with external input devicesor external output devices

The neural network can be represented and built/constructed based on theneurons and synaptic associations, with appropriate rules for creatingthe neural structure and traversing the neural connections. The networklearns by creating associations between neurons. For any neuron, itsrepresentation can be recalled and expressed, and exploration ofmultiple associations and permutations of its associations, with otherneurons can be recalled or expressed. Neurons can represent memories,where a memory is defined as the expression of a neuron.

Interfacing with or learning or processing an experience of externalsensory neuron events and creating memories of input experiences isrepresented by neurons and by new connections between neurons.

Interfacing with or learning or representing an activity of externalmotor neuron events and creating memories of output actions isrepresented by new neurons and by new connections between neurons.

The neural network can learn, it can express or recall or recollectmemories, it can interact or express itself with the externalenvironment, and has the ability to think (a process defined asexploring alternative neuron associations as described above) and toexpress itself creatively.

The present invention also extends to a computer usable mediumcomprising a computer program code or specially built digital processorchips customised to cause one or more processors to execute one or morefunctions to perform the methods described above.

Whilst there has been described in the foregoing description preferredembodiments of the present invention, it will be understood by thoseskilled in the technology that many variations or modifications isdetails of design, construction or operation may be made withoutdeparting from the present invention.

1. An artificial neural network comprising: (a) a plurality of neurons,(b) each of the plurality of neurons being a processor with memory andbeing in an array; (c) the plurality of neurons comprising a pluralityof elemental neurons and a plurality of structural neurons; (d) allelemental and structural neurons being configured to be associated withothers of the elemental and structural neurons via active connections;(e) each elemental neuron being configured to: (i) represent a uniqueinput value into the artificial neural network, the unique value beingat least one selected from the group consisting of: a stimulus, anevent, events, a sequence in a pattern, a sequence of events, anelemental stimulus, a defined elemental pattern, a defined elementaldata element, a basic input stimulus, and an output stimulus ofinformation being processed; and (ii) express that unique value as anoutput when activated by a structural neuron; (f) each structural neuronbeing configured to: (i) receive input from a pair of neurons of theplurality of neurons and with which it is an associating neuron; and(ii) express that input as an output to that pair of neurons to activatethe pair of neurons for expression.
 2. An artificial neural network asclaimed in claim 1, wherein any one of the plurality of neurons is ableto associate with a neuron in the plurality of neurons via the activeconnections to a further neuron in the plurality of neurons, the furtherneuron being one of the plurality of structural neurons.
 3. Anartificial neural network as claimed in claim 1, wherein each structuralneuron represents the combined information of patterns by the pair ofneurons with which it associates, the structural neuron receiving inputfrom the pair of neurons.
 4. An artificial neural network as claimed inclaim 1, wherein the plurality of elemental neurons is in a root levelof the neural network.
 5. An artificial neural network as claimed inclaim 1, wherein the pair of neurons comprises at least one selectedfrom the group consisting of: an elemental neuron and an elementalneuron, an elemental neuron and a structural neuron, a structural neuronand an elemental neuron, and a structural neuron and a structuralneuron.
 6. An artificial neural network as claimed in claim 1, whereineach of the plurality of neurons is one or more selected from the groupconsisting of: initiating neuron, associated neuron, and associatingneuron; an initiating neuron being associated with an associated neuronvia connections to the associating neuron.
 7. An artificial neuralnetwork as claimed in claim 6, wherein the initiating neuron, theassociated neuron and the associating neuron are connected based onproximal characteristics, the proximal characteristics being at leastone of: temporal, spatial, intensity, magnitude and relative position ofthe input being processed.
 8. An artificial neural network as claimed inclaim 6, wherein each initiating neuron is able to associate with aplurality of associated neurons to form a plurality of pairs of neurons.9. An artificial neural, network as claimed in claim 6, wherein eachassociated neuron is able to associate with a plurality of initiatingneurons to form a plurality of pairs of neurons.
 10. An artificialneural network as claimed in claim 6, wherein when an initiating neuronreceives input and an associated neuron receives input, the inputs aretransmitted to all associating neurons of the initiating neuron and theassociated neuron, the associating neuron of both the initiating neuronand the associated neuron then being activated and being able to produceoutput.
 11. An artificial neural network as claimed in claim 10, whereinthe associated neuron is activated and able to produce output in amanner selected from the group consisting of: at the same time as theinitiating neuron, and after the initiating neuron.
 12. An artificialneural network as claimed in claim 10, wherein the activation orproduction of output of the initiating neuron and the associated neuronis also based on proximal characteristics.
 13. An artificial neuralnetwork as claimed in claim 12, wherein the proximal activation of orproduction of output from the initiating neuron and the associatedneuron causes at least one selected from the group consisting of: thecreation of a new associating neuron if none exists together with newconnections between the initiating neuron and the new associating neuronand between the associated neuron and the new associating neuron, thestrengthening of existing connections between the initiating neuron andthe associating neuron and between the associated neuron and theassociating neuron, and strengthening of the associating neuron.
 14. Anartificial neural network as claimed in claim 13, wherein thestrengthening is by maintaining a frequency count of how often theassociating neuron receives input from the initiating neuron and theassociated neuron.
 15. An artificial neural network as claimed in claim6, wherein the associating neuron represents the sum of what isrepresented by the initiating neuron and the associated neuron.
 16. Anartificial, neural network as claimed in claim 6, wherein once theassociating neuron represents a result, the result need not be createdin another neuron.
 17. An artificial neural network as claimed in claim1, wherein the plurality of elemental neurons is configured to: receiveall input to the artificial neural network, and provide all output fromthe artificial neural network.
 18. An artificial neural network asclaimed in claim 1, wherein all neurons represent at least one of:value, information and pattern; and processing is at least one of:associating neurons, expressing the pair of neurons with which astructural neuron associates, and expressing the value information orpattern represented by elemental neurons.
 19. An artificial neuralnetwork as claimed in claim 1, wherein a level of the neural network isa deeper level within the neural network structure if, duringexpression, more steps are required to express the elemental neuronsthat it represents.
 20. An artificial neural network as claimed in claim17, wherein associating a pair of neurons is learning, and expressing apair of neurons is expression.
 21. An artificial neural network asclaimed in claim 19, wherein the artificial neural network isbi-directional with a forward mode being learning, and a reverse modebeing expression.
 22. An artificial neural network as claimed in claim1, wherein the artificial neural network stores associations and notinput data and represents patterns within patterns of associations. 23.An artificial neural network as claimed in claim 1, wherein eachelemental neuron is selected from the group consisting of: a sensorneuron and a motor neuron.
 24. An artificial neural network as claimedin claim 1, wherein each structural neuron represents a plurality ofelemental neurons.
 25. An artificial neural network as claimed in claim1, wherein each of the plurality of neurons is able to be expressed. 26.An artificial neural network as claimed in claim 1, wherein the numberof elemental neurons and structural neurons required for the memory isdetermined by the nature of the input to be processed.
 27. An artificialneural network as claimed in claim 1, wherein the memory is to store afrequency of received inputs.
 28. An artificial neural network asclaimed in claim 6, wherein each neuron is a node in the array, eachnode having a plurality of pointers.
 29. An artificial neural network asclaimed in claim 28, wherein the plurality of pointers comprises twopointers for providing expression and further pointers to representassociations.
 30. An artificial neural network as claimed in claim 28,wherein each pointer in each node contains at least one selected fromthe group consisting of: an address of another neuron, an elementalvalue for an elemental neuron, and a frequency count.
 31. An artificialneural network as claimed in claim 28, wherein the number of pointersdepends on a function being performed by the artificial neural network,the number of pointers for each neuron being at least two.
 32. Anartificial neural network as claimed in claim 28, wherein a function ofeach pointer to a neuron is selected from the group consisting of:initiating, associating, successor, next successor of the initiatingneuron, precessor, and next precessor of the associating neuron.
 33. Anartificial neural network as claimed in claim 28, wherein at least onepointer for an elemental neuron represents elemental values.
 34. Anartificial neural network as claimed in claim 28, wherein all neuronsare a fixed length addressable node in the array.
 35. An artificialneural network as claimed in claim 1, wherein the artificial neuralnetwork is used for at least one selected from the group consisting of:monitoring and predicting stock price movements, Internet surveillance,Internet security, computer virus detection, computer spam detection,phrases in speech and text, clauses in speech and text, plagiarismdetection, bioinformatics, vision recognition, semantic analysis,representation of data ontologies, robotics, and data compression. 36.An artificial neural network comprising: (a) a plurality of neurons, (b)each of the plurality of neurons being a processor with memory and beingin an array; (c) the plurality of neurons comprising a plurality ofelemental neurons and a plurality of structural neurons; (d) allelemental and structural neurons being configured to be associated withothers of the elemental and structural neurons via active connections;(e) each elemental neuron being configured to: (i) represent a uniquevalue able to be input into the artificial neural network system, theunique value being one of: a stimulus, an event, events, a sequence in apattern or sequence of events; and (ii) express that unique value as anoutput; and (f) each structural neuron being configured to receive inputfrom a pair of neurons with which it is associating, the pair of neuronsbeing selected from the group consisting of: both elemental neurons,both structural neurons, one structural and one elemental neuron, andone elemental neuron and one structural neuron.
 37. An artificial neuralnetwork as claimed in claim 36, wherein the plurality of elementalneurons are in a root level of the neural network, and each elementalneuron represents a unique value, the unique value being at least oneselected from the group consisting of: a stimulus, an event, events, asequence in a pattern, a sequence of events, an elemental stimulus, adefined elemental pattern, a defined elemental data element, a basicinput stimulus, and an output stimulus of information being processed.38. An artificial neural network as claimed in claim 36, wherein eachelemental neuron is selected from the group consisting of: a sensorneuron and a motor neuron.
 39. An artificial neural network as claimedin claim 36, wherein all neurons represent at least one of: value,information and pattern; and processing is at least one of: associatingneurons, expressing the pair of neurons with which a structural neuronassociates, and expressing the value information or pattern representedby elemental neurons.
 40. An artificial neural network as claimed inclaim 36, wherein neuron associations are represented in a plurality ofdeeper neural levels; the number of levels in the plurality of deeperlevels being determined by the extent of the pattern to be processed orexpressed, where a structural neuron represents a plurality of elementalneurons.
 41. An artificial neural network as claimed in claim 40,wherein the number of elemental neurons and structural neurons requiredfor the memory is determined by the nature of the input to be processed.42. An artificial neural network as claimed in claim 36, wherein any oneof the plurality of neurons is able to associate with a neuron in theplurality of neurons via the active connections to a further neuron inthe plurality of neurons, the further neuron being one of the pluralityof structural neurons.
 43. An artificial neural network as claimed inclaim 36, wherein the artificial neural network is bi-directional with aforward mode being learning, and a reverse mode being expression.
 44. Anartificial neural network as claimed in claim 36, wherein the artificialneural network stores associations and not input data and representspatterns within patterns of associations.
 45. An artificial neuralnetwork as claimed in claim 36, wherein each of the plurality of neuronsis able to be expressed.
 46. An artificial neural network comprising:(a) a plurality of neurons, (b) each of the plurality of neurons being aprocessor with memory and being in an array; (c) the plurality ofneurons comprising a plurality of elemental neurons and a plurality ofstructural neurons; (d) all elemental and structural neurons beingconfigured to be associated with others of the elemental and structuralneurons via active connections; (e) each elemental neuron beingconfigured to: (i) represent a unique value able to be input into theartificial neural network system, the unique value being at least oneselected from the group consisting of: a stimulus, an event, events, asequence in a pattern, a sequence of events, an elemental stimulus, adefined elemental pattern, a defined elemental data element, a basicinput stimulus, and an output stimulus of information being processed;and (ii) express that unique value as an output; (f) all of theplurality of structural neurons being able to be expressed in terms ofthe elemental neurons from which they were derived or represent.
 47. Anartificial neural network as claimed in claim 46, wherein the artificialneural network is bi-directional with a forward mode being learning, anda reverse mode being expression.
 48. An artificial neural network asclaimed in claim 46, wherein the artificial neural network storesassociations and not input data and recognizes patterns within patternsof associations.
 49. An artificial neural network as claimed in claim46, wherein the neural network is bi-directional with all elementalneurons being able to express their elemental values, and all structuralneurons being able to express a pair of neurons with which theyassociate.
 50. An artificial neural network comprising: (a) a pluralityof neurons, (b) each of the plurality of neurons being a processor withmemory and being a node in an array; (c) the plurality of neuronscomprising a plurality of elemental neurons and a plurality ofstructural neurons; (d) all elemental and structural neurons beingconfigured to be associated with others of the elemental and structuralneurons via connections; (e) the artificial neural network beingbi-directional and being able to operate in a forward mode wherestructural neurons are created from input events from the elementalneurons, and in a reverse mode where input events are expressed by theelemental neurons.
 51. An artificial neural network as claimed in claim50, wherein the forward mode is learning, and the reverse direction isexpression.
 52. An artificial neural network as claimed in claim 50,wherein the neural network stores associations and not input data. 53.An artificial neural network as claimed in claim 50, wherein the neuralnetwork represents and recognizes patterns within patterns ofassociations.
 54. A neuronal assembly of an artificial neural network,the neuronal assembly comprising an initiating neuron, an associatedneuron, and an associating neuron operatively connected with theinitiating neuron and the associated neuron; the associating neuronrepresenting the sum of what is represented by the initiating neuron andthe associated neuron, and once the associating neuron represents aresult, the result need not be created in another neuron.
 55. A neuronalassembly as claimed in claim 54, wherein the artificial neural networkis as claimed in claim
 1. 56. A neuronal assembly as claimed in claim54, wherein when an initiating neuron receives input and an associatedneuron receives input, the inputs are transmitted to all associatingneurons of the initiating neuron and the associated neuron, theassociating neuron of both the initiating neuron and the associatedneuron then being activated and being able to produce output.
 57. Aneuronal assembly as claimed in claim 56, wherein the associated neuronis able to produce output in a manner selected from the group consistingof: at the same time as the initiating neuron, and after the initiatingneuron.
 58. A neuronal assembly as claimed in claim 56, wherein theactivation of or producing of output from of the initiating neuron andthe associated neuron is based on proximal characteristics.
 59. Aneuronal assembly as claimed in claim 58, wherein the proximalactivation of or producing of output from the initiating neuron and theassociated neuron causes at least one selected from the group consistingof: the creation of a new associating neuron if none exists togetherwith new connections between the initiating neuron and the newassociating neuron and between the associated neuron and the newassociating neuron, the strengthening of existing connections betweenthe initiating neuron and the associating neuron and between theassociated neuron and the associating neuron, and strengthening of theassociating neuron.
 60. A method for creating an association of neuronsin an artificial neural network having a plurality of neurons, one ofthe plurality of neurons being an initiating neuron, another of theplurality of neurons being an associated neuron operatively connectedwith the initiating neuron, and a further neuron of the plurality ofneurons being an associating neuron operatively connected with theinitiating neuron and the associated neuron; the method comprising:activating or producing an output from the initiating neuron topotentiate the associating neuron; and activating or producing an outputfrom the associated neuron to potentiate and activate the associatingneuron, the associating neuron then being activated and able to producean output; the associating neuron representing the sum of what isrepresented by the initiating neuron and the associated neuron, and oncethe associating neuron represents a result, the result need not becreated in another neuron.
 61. A method as claimed in claim 60, whereinthe associating neuron is expressed by the associating neuron expressingthe initiating neuron and the associated neuron.
 62. A method as claimedin claim 60, wherein the associated neuron is activated or produces anoutput in a manner selected from the group consisting of: at the sametime as the initiating neuron, and after the initiating neuron.
 63. Amethod as claimed in claim 60, wherein the activation of or producing anoutput by the initiating neuron and the activation of or producing anoutput by the associated neuron is based on proximal characteristics.64. A method as claimed in claim 63, wherein the proximal activation ofor producing output from the initiating neuron and the associated neuroncauses at least one selected from the group consisting of: the creationof a new associating neuron if none exists together with new connectionsbetween the initiating neuron and the new associating neuron and betweenthe associated neuron and the new associating neuron, the strengtheningof existing connections between the initiating neuron and theassociating neuron and between the associated neuron and the associatingneuron, and strengthening of the associating neuron.
 65. A method asclaimed in claim 60, wherein the associating neuron represents the sumof what is learnt from the initiating neuron and the associated neuron.66. A method as claimed in claim 64, wherein once the new associatingneuron is created to represent a result, the result need not be createdin another neuron.
 67. A method as claimed in claim 60, wherein once theassociating neuron represents a result, the result need not be createdin another neuron.
 68. A method of constructing an artificial neuralnetwork comprising a plurality of neurons, the plurality of neuronscomprising a plurality of elemental neurons and a plurality ofstructural neurons, the method comprising: defining unique events theelemental neurons will represent; creating a required number ofelemental neurons for the total number of unique values to berepresented for all defined events; the unique value being at least oneselected from the group consisting of: a stimulus, an event, events, asequence in a pattern, a sequence of events, an elemental stimulus, adefined elemental pattern, a defined elemental data element, a basicinput stimulus, and an output stimulus of information being processed;the plurality of elemental neurons receiving all input to the artificialneural network, all output from the artificial neural network being fromthe plurality of elemental neurons; creating the plurality of structuralneurons, each of the structural neurons being created by the associationof a pair of the plurality of neurons; each of the plurality ofstructural neurons being configured to produce an output on activationby the pair of neurons, the pair of neurons comprising an initiatingneuron and an associated neuron; the association of the plurality ofneurons being based on proximal characteristics; and each of theplurality of structural neurons being configured to express the pair ofneurons.
 69. A method as claimed in claim 68, wherein any one of theplurality of neurons is able to associate with a neuron in the pluralityof neurons via active connections to a further neuron in the pluralityof neurons, the further neuron being one of the plurality of structuralneurons.
 70. A method as claimed in claim 68 wherein all elementalneurons are able to express their elemental values, and all structuralneurons are able to express a the pair of neurons with which theyassociate.
 71. A method as claimed in claim 70, wherein the pair ofneurons is selected from the group consisting of: an elemental neuronwith an elemental neuron, an elemental neuron with a structural neuron,a structural neuron with an elemental neuron, a structural neuron with astructural neuron.
 72. A method as claimed in claim 68, wherein each ofthe plurality of neurons is one or more selected from the groupconsisting of: initiating neuron, associating neuron, and associatingneuron; an initiating neuron being associated with an associated neuronvia connections to the associating neuron.
 73. A method as claimed inclaim 72, wherein the initiating neuron, the associated neuron and theassociating neuron are connected based on proximal characteristicsselected from the group consisting of: temporal, spatial, intensity,magnitude and relative position of the input being processed.
 74. Amethod as claimed in claims 68, wherein a level of the neural network isa deeper level within the artificial neural network if, duringrecollection, more steps are required to express the elemental neurons.75. A method as claimed in claim 72, wherein when an initiating neuronreceives input and an associated neuron receives input, the inputs aretransmitted to all associating neurons of the initiating neuron and theassociated neuron respectively, the associating neuron of both theinitiating neuron and the associated neuron then being activated andbeing able to produce output.
 76. A method as claimed in claim 75,wherein the associated neuron is activated or an output produced in amanner selected from the group consisting of: at the same time as theinitiating neuron, and after the initiating neuron.
 77. A method asclaimed in claim 75, wherein the activation of or producing an outputfrom the initiating neuron and the activation of or producing an outputfrom the associated neuron is based on proximal characteristics.
 78. Amethod as claimed in claim 77, wherein the proximal activation of orproducing an output from the initiating neuron and the associated neuroncauses at least one selected from the group consisting of: the creationof a new associating neuron including new synaptic connections betweenthe initiating neuron and the new associating neuron and between theassociated neuron and the new associating neuron, the strengthening ofexisting synaptic connections between the initiating neuron and theassociating neuron and between the associated neuron and the associatingneuron, and the strengthening of the associating neuron.
 79. A method asclaimed in claim 78, wherein the strengthening is by maintaining afrequency count of how often the associating neuron receives input fromthe initiating neuron and the associated neuron.
 80. A method as claimedin claim 72, wherein the associating neuron represents the sum of whatis represented by the initiating neuron and the associated neuron.
 81. Amethod as claimed in claim 68, wherein a memory represents a pluralityof elemental stimuli, and each elemental stimulus is representeddirectly by an elemental neuron.
 82. A method as claimed in claim 68,wherein the number of elemental neurons required to represent the memoryis determined by the nature of the input being processed.
 83. A methodas claimed in claim 68, wherein each neuron is represented by anaddressable node in an array, each node having a plurality of pointers.84. A method as claimed in claim 68, wherein the plurality of elementalneurons is in a root level of the neural network.
 85. A method asclaimed in claim 72, wherein each initiating neuron is able to associatewith a plurality of associated neurons to form a plurality of pairs ofneurons.
 86. A method as claimed in claim 72, wherein each associatedneuron is able to associate with a plurality of initiating neurons toform a plurality of pairs of neurons.
 87. A method as claimed in claim72, wherein once the associating neuron represents a result, the resultneed not be created in another neuron.
 88. A method as claimed in claim68, wherein the plurality of elemental neurons is configured to: receiveall, input to the artificial neural network, and provide all output fromthe artificial neural network.
 89. A method as claimed in claim 68,wherein all neurons represent at least one of: value, information andpattern; and processing is at least one of: associating neurons,expressing the pair of neurons with which a structural neuronassociates, and expressing the value information or pattern representedby elemental neurons.
 90. A method as claimed in claim 68, whereinassociating a pair of neurons is learning, and expressing a pair ofneurons is expression.
 91. A method as claimed in claim 90, wherein theartificial neural network is bi-directional with a forward mode beinglearning, and a reverse mode being expression.
 92. A method as claimedin claim 68, wherein the artificial neural network stores associationsand not input data and represents patterns within patterns ofassociations.
 93. A method as claimed in claim 68, wherein eachelemental neuron is selected from the group consisting of: a sensorneuron and a motor neuron.
 94. A method as claimed in claim 68, whereineach structural neuron represents a plurality of elemental neurons. 95.A method as claimed in claim 68, wherein each of the plurality ofneurons is able to be expressed.
 96. A method as claimed in claim 83,wherein the plurality of pointers comprises two pointers for providingexpression and further pointers to represent associations.
 97. A methodas claimed in claim 96, wherein each pointer in each node contains atleast one selected from the group consisting of: an address of anotherneuron, an elemental value for an elemental neuron, and a frequencycount.
 98. A method as claimed in claim 96, wherein the number ofpointers depends on a function being performed by the artificial neuralnetwork, the number of pointers for each neuron being at least two. 99.A method as claimed in claim 96, wherein a function of each pointer to aneuron is selected from the group consisting of: initiating,associating, successor, next successor of the initiating neuron,precessor, and next precessor of the associating neuron.
 100. A methodas claimed in claim 96, wherein at least one pointer for an elementalneuron represents elemental values.
 101. A method as claimed in claim96, wherein all neurons are a fixed length addressable node in thearray.
 102. A method as claimed in claim 68, wherein the artificialneural network is used for at least one selected from the groupconsisting of: monitoring and predicting stock price movements, Internetsurveillance, Internet security, computer virus detection, computer spamdetection, phrases in speech and text, clauses in speech and text,plagiarism detection, bioinformatics, vision recognition, semanticanalysis, representation of data ontologies, robotics, and datacompression.
 103. A computer usable medium comprising a computer programcode configured to cause one or more processors and/or memory to executeone or more functions to perform the method claimed in claim
 60. 104. Acomputer usable medium comprising a computer program code configured tocause one or more processors and/or memory to execute one or morefunctions to perform the method claimed in claim 68.